Information free energy for nonequilibrium states 
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We introduce an information free energy for thermodynamic systems driven by external time- 
dependent parameters. We show that the latter is a nonequilibrium state function and that it 
is a natural generalization of the usual equilibrium and steady state free energies. We discuss 
its importance for the nonequilibrium maximum work theorem and the Jarzynski relation in the 
presence of feedback control. We further show that it is a nonequilibrium Lyapunov function. 
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Thermodynamics is a theory of processes and states. 
Thermodynamic functions are accordingly divided into 
process-dependent quantities, like work and heat, and 
state variables that do not depend on specific transfor- 
mations, but only on the current state of a system [T]. 
Contrary to work and heat, thermodynamic state func- 
tions are only defined at equilibrium. Two important 
examples of equilibrium variables are the free energy F 
and the entropy S, which are both related to work or 
heat for quasistatic processes: the isothermal change of 
free energy is equal to the mean work, AF — (W) , while 
the change of entropy is given by the mean heat divided 
by temperature, AS — (Q) /T. A central question is 
how to extend these well-established concepts to arbi- 
trary processes and nonequilibrium states. The Shannon 
information entropy of a probability distribution p{x) 
is defined as % — — J dxp(x) lnp(x) [I]. Remarkably, 
for equilibrium transformations both entropies coincide, 
AH = AS, and the variation of information entropy is 
thus also determined by the mean heat [3] (we here set the 
Boltzmann constant to unity). This identity illustrates 
the intimate connection between thermodynamics and in- 
formation theory. The Shannon entropy %, being defined 
for any probability distribution, is commonly used as a 
nonequilibrium entropy [1HZ]- However, its precise rela- 
tionship to the mean heat out of equilibrium seems to 
be unclear. Moreover, a proper generalization of the free 
energy valid far from equilibrium appears to be lacking. 

In the present article, we introduce an information 
based free energy T that is a nonequilibrium state func- 
tion. We show that it reduces to the usual free energy 
for equilibrium states, and to the free energy of Hatano 
and Sasa for nonequilibrium steady states [8]. We, more- 
over, highlight its essential role in the formulation of 
the nonequilibrium maximum work theorem _1], and in 
the generalized work relation under nonequilibrium feed- 
back control [5]. In addition, we establish that it decays 
monotonously during relaxation processes towards equi- 
librium or nonequilibrium steady states, and that it is, 
hence, a proper nonequilibrium Lyapunov function |10j . 



Information free energy. In the following, we consider 
the paradigmatic model of an overdamped Brownian 
particle moving in a time-dependent potential V(x, a), 
where a = at is an external control parameter. The time 
evolution of the particle is described by the Smoluchowski 
equation for the probability density p(x,t) |llj . 



d t p{x,a) = L a p(x,a) 



with L a = — d x 

7 



V'(x,a) + -d x 



(1) 



where 7 is the damping coefficient and f3 the inverse tem- 
perature. For a particular value of the parameter cv t , the 
stationary state verifies dtp(x,ctt) = L a p(x,at) = 0. 
We begin by treating the case of equilibrium steady 
states and investigate the connection between informa- 
tion entropy and heat in the quasistatic limit. For 
quasistatic driving a t — > 0, we have d t p(x,a t ) = 
J dta t d a p(x,at) —> 0. We note, however, that for a 
general (nonequilibrium) distribution, L a p{x,ctt) 7^ 0. 
Let us, now, consider the difference between the change 
of the Shannon entropy and the mean heat, 



(£) = AH - (3 (Q) 



(2) 



The quantity (E) may be regarded as an information en- 
tropy production [B]. The average heat exchanged with 
the bath is given by the usual definition [12] . 



P{Q) = J ^ J dxd tP (x,a t )l3V(x,a t ). (3) 

At the same time, the variation of the Shannon entropy 
can be written in the form, 

AW = — J dt J dx d t p(x,a t ) In p(x,a t ) ■ (4) 

Combining Eqs. ^ and Q, and using the Smoluchowski 
equation ([I]), we obtain after integrations by parts, 



dt / da: 



(p'(x,a t ) + l3V'(x,at)p(x,a t )Y 
f3jp(x,a t ) 



(5) 
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The above expression shows that AW = (3 (Q) only if the 
current j(x, at) = p'(x, at) + /3V'(x, at)p{x, a t ) = 0, i.e. 
if the system is in a stationary state, L a p(x,at) = 0, at 
all times. The latter condition is fulfilled for d t — > and 
initial stationary states L ao p(x,ao) — 0. We can, thus, 
conclude that for arbitrary nonequilibrium initial states, 
the change of information entropy is not given by the 
mean heat, even for quasistatic driving. The latter can 
be understood by noticing that an initial nonequilibrium 
state will relax spontaneously to the equilibrium state, 
and the relaxation time is in general different from the 
driving time. Thus, relaxation needs not be slow, even if 
the driving is. As a result, the mean information entropy 
production is strictly positive in that instance, (S) > 0. 

Equation ^ has important consequences for the max- 
imal amount of work that can be extracted from a 
given system. Using the first law of thermodynamics, 
AU = U(a T ) - U(a ) = (W) + (Q), with U(a) = 
J dxp(x, a) V(x, a), we find, 



(E) = AH - P (Q) = AH - P AU + p (W) 
= (3(W)- PAT > , 



(6) 



where we have introduced the information free energy, 



\T{a t )=pU{at)-U{a t ) 

= D(p(x,a t )\\p cq (x,a t )) 



-PF(at) 



Here D (p(x,a)\\p cq (x 1 a)) = J dxp(x, a) hip(x, a) — 
J dxp(x, a) lnp cq (x, a) is the relative (Kullback-Leibler) 
entropy [12] [T3] between p(x, a) and the equilibrium dis- 
tribution p e q(x, a) ~ exp (— PV(x, a))/Z(a) correspond- 
ing to the same parameter a. The equilibrium free en- 
ergy reads F(a) — — (1//3) In Z(a), where Z(a) is the 
partition function. The information free energy ([7| is a 
nonequilibrium state function, since for a cyclic process 
between arbitrary nonequilibrium states, 



<j> dF = F(a ) — F(a ) =0, a$ = a T . 



(8) 



The information free energy F is strictly speaking a rel- 
ative quantity that depends both on the actual state of 
the system, p(x, a t ) and the reference equilibrium state, 
p eq {x, at). However, by considering the latter to be fixed 
(for given potential and temperature), F can be inter- 
preted as depending on the arbitrary state p(x, at) alone. 
Equation ^ is an expression of the maximum work theo- 
rem, extended to arbitrary nonequilibrium states |15[I16|: 
the maximum work, — (W), that can be extracted from a 
system is always smaller than the change of information 
free energy, —AF. The bound is tight only for stationary 
states in the quasistatic limit. 

The information free energy ^ is defined for arbi- 
trary nonequilibrium states. For transitions between two 
equilibrium states, p eq (x,a') and p eq (x,a), the relative 
entropy simplifies to D (p cq (x,a')\\p cq (x,a)) = F(a') — 



F(a) [IT] , and F(a') reduces to the equilibrium free en- 
ergy F(a'). On the other hand, for transitions between 
two nonequilibrium steady states, p ss {x, a') and p ss (x, a), 
the information free energy reduces to the steady state 
free energy defined by Hatano and Sasa [5]. 

We may indeed repeat the above analysis for nonequi- 
librium steady states induced by either a driven potential 
[18] or a nonconservative force ft [13 EQ] by considering, 
in Eq. ([T]), the operator 



9.r 



V'(x,a) + f t + -d x 



(9) 



In this case, following Hatano and Sasa [8] (see also 
Ref . [21] ) , we divide the total heat in house- keeping heat 
(exchanged in a given nonequilibrium steady state) and 
excess heat (exchanged during a change of steady states) , 
Qtot = Qhk + Qcx- For equilibrium steady states, we sim- 
ply have Q to t = Qex- The excess heat reads [S], 



P(Qex) = J dt J dxp(x, at)xd x ip(x,a t ) , (10) 

where ip(x, at) = — In p ss {x 7 at) and p ss (x,at) is the 
nonequilibrium steady state, L a p ss {x, a) — 0. The dif- 
ference between the information entropy change and the 
(7) mean excess heat, (S ox ) = AH — /3(Q ox ), can then be 



determined as done previously. We find 



dt / dx 



j(x,a t ) 2 
Pjp(x,a t ) 



(11) 



Accordingly, AU = P (Q cx ) only if j(x, a t ) = p'(x, a t ) + 
P(V'(x, a t ) + ft) p(x,a t ) — at all times, that is only 
for quasistatic processes starting in initial stationary 
states. Introducing the excess work as (W ex ) = AU — 
(Qox) and the information free energy Q, the maximal 
work theorem takes here the form, 



(Wcx) > AT . 



(12) 



Equation ( 12 ) generalizes the inequality obtained by 



Hatano and Sasa [S] to arbitrary initial nonequilibrium 
states. Again, the bound is tight only for initial station- 
ary states and quasistatic driving. 

Nonequilibrium feedback control. The information 
free energy ([7| finds also application in the context of 
feedback control. The Jarzynski equality (exp(— PW)) = 
exp(-pAF) [22] has been extended by Sagawa and Ueda 
to account for external information gain [9]. The latter 
result is however restricted to initial equilibrium states, 
and a generalization to arbitrary states is therefore of in- 
terest [21] ■ Following Ref. [53] , we consider the situation 
where the parameterization of the control parameter a is 
updated during the process by feedback measurements. 
We denote by X = {x t }J a trajectory of the system that 
starts at t = and ends at t = r, and suppose that a 
measurement on X is performed at time t. Let x t be 
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the space point of the system, P[xt] its probability, and 
y the measurement outcome — we thus have a = a(t,y). 
We further assume that the measurement can involve an 
error which is described by the conditional probability, 
P[j/|a;t], to measure y while the system is at position Xt- 
The probability of the measurement outcome y is then, 



P[y] 



dx t P[y\x t }P[x t } . 



(13) 



The information obtained from the measurement is char- 
acterized by the mutual information [9], 



(I) 



where I[xt, \y] is given by 

I[x t> y] = In 



dx t / dyP[y\x t }P[x t }I[x tl y}, (14) 



P\v\ 



(15) 



Following the path integral approach elaborated in 
Refs. [2H1 HZ] i we introduce the probability of a forward 
trajectory P F [X\. The time reversed counterpart is ac- 
cordingly P R [X^] = exp(-£)P F [A] [261E7]. As a result, 



P[y) 



(exp (-E - /)) = 

= jvxj dyP[y\x t }P F [X] exp (-S) ^-^ (lg) 

= y^xt y dyp« [jrt] p[ y ]=i, 



where we have used Eq. (151 and the fact that VX = 



T>X^ . The last equality follows from normalization. Ex- 



pression ( 16 1 is a generalization of the Jarzysnki equality 



with feedback to initial nonequilibrium states. An appli- 
cation of Jensen's inequality yields, 



(£) + <J)>0. 



(17) 



Making eventually use of the explicit expression of the 
entropy production in a nonequilibrium steady state, £ = 

= fidtd a (pa, 1251122, 

we obtain, 



(S 



dt d a ip a 



dt / dxp(x, t) x d x <f 



= AH - j3 (Q ex ) - P (W ex > - /3AJ\ 



(18) 



The generalization of the second law to nonequilibrium 
steady states with feedback takes therefore the form, 



P (Wex) > (3 AT - (I) . 



(19) 



A similar calculation for arbitrary nonequilibrium states 
leads to a general result of the same form for (W), show- 
ing that more work can be extracted from a system in the 
presence of feedback [55] . The latter reduces to the spe- 
cial result derived by Sagawa and Ueda for equilibrium 
states when JF is replaced by F. 



Nonequilibrium Lyapunov function. Equilibrium 
states are stable and correspond to a minimum of the 
free energy pQ. The relaxation of a nonequilibrium 
perturbation back to the equilibrium state is often 
characterized by the monotonic decay of a Lyapunov 
function. The question of the existence of Lyapunov 
functions for the relaxation towards general nonequilib- 
rium steady states has recently been raised [ID] . It is 
often believed that the free energy difference should be 
replaced by the relative entropy away from equilibrium. 
However, due to its oscillating behavior, the latter is 
not a Lyapunov function [10] . The central point of 
the present paper is that the correct nonequilibrium 
generalization of the free energy is the information free 
energy ([7]). For purely relaxation processes towards a 
steady state ((W cx ) — 0), its time derivative is simply, 



id t ^(a) 



-d t (S ex ) < 0. 



(20) 



The information free energy hence decays monotonously 
to its minimal value at all times — and not only asymptot- 
ically (see Ref. [10J). It is therefore a true Lyapunov func- 
tion for equilibrium and nonequilibrium steady states. 

Examples. To illustrate the results ([6]), (12 1 and (20) 
for equilibrium and nonequilibrium steady states, we first 
consider a Brownian particle in a harmonic trap with 
time-dependent frequency, V{x,a t ) — a t x 2 /2. This sys- 
tem has been studied experimentally to verify the fluc- 
tuation theorem [25] • For concreteness, we choose the 
initial (nonequilibrium) distribution p(x, 0) to be Gaus- 
sian. Due to the linearity of the Smoluchowski equation 
([T]), its time-dependent solution p(x,t) is also Gaussian. 
Its mean \x t and variance o~ t satisfy the equations, 



2& t 2 1 a t 

Mi = I o 2 H J M*' 

V a t PI Ot 7 / 

11 a t 

<?t = ~n Of 

P7 o t 7 

The corresponding time-dependent solutions are 



1 



ds a. 



A*t — Mo ex P | / ^ w s | > 

7 Jo 



a\ = <7q exp ( J cLmi, 



(21a) 
(21b) 

(22a) 



+ ^/ dsexp ( v 

o 



y ds'a s >\ (22b) 

The mean heat ^ exchanged with the bath follows as, 
0{Q)=P [ dt a t (fit fit + o t o t ) , (23) 



and the variation of information entropy Q reads 



AH = -\ ln4- 
2 al 



(24) 
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FIG. 1: (color online) Average excess work (W ex ) (red, solid) 
and information free energy difference AJ-" (blue, dashed) as 
a function of the driving time r for a particle in the tilted 
periodic potential (261, starting in the stationary state (28 1. 
Parameters are 7 = 1, /3 = 25, k = 1, ho = 0.2, and hi = 0.6. 



On the other hand, the mean work is 



(W) = dt dxp{x,t)d a V a t 

ir (25) 
= 2 J At6lt ( M * +cr *) ' 

and the information free energy difference is given by 



AT 



2 



04 



2 



1 



In 



(26) 



We notice that these two last expressions usually differ. 
In the quasistatic limit, a T ~ (a r — o.q)/t — ¥ 0, the first 
two terms on the right-hand side of Eq. ( 26 1 reduce to 
the mean work (25 1. However, the last term in Eq. (26) 



does not depend on the driving rate and only vanishes 
if <r 2 = <7q, i.e. when the system already starts in the 
equilibrium state p(x, 0) = p cq (x, 0). The time derivative 
of the information free energy ^ further reads 



d t T{a t ) 



I 



dxp(x, a t ) 1/3 a t x 



fit 



(27) 

which is clearly strictly negative at all times. 

To examine the case of a nonequilibrium steady state, 
we next consider a particle in a tilted periodic potential, 
V(x, t) = — fc cos x — xh(t), with h(t) = h — (h — hi)t/r. 
The stationary solution of Eq. ([!]), with periodic bound- 
ary conditions and V'(x) — V'(x + 2ir), is [2"9"j . 



Ps{x) 



exp(-pV{x,t)) 



x+2ir 



dx exp (PV(x, t)), 



(28) 

where Nt is the normalization constant. Figure [TJ shows 
the mean excess work (W cx ) and the information free 
energy difference AT as a function of the duration of the 
process r, when the particle is initially in the stationary 
state (28). As expected, in the quasistatic limit, r — > 00, 
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FIG. 2: (color online) Average excess work (Wex) (red, solid) 
and information free energy difference (blue, dashed) as 
a function of the driving time r for a particle in the tilted 
periodic potential (26 1, starting in a nonstationary Gaussian 
state (see text). Parameters are ao = 0.25, 7=1,/? = 25, 
k — 1, ho = 0.2, and hi — 0.6. The inset shows the monotonic 
decay of the information free energy J as a function of time 
for the same initial state, but no external driving. 



(Wex) approaches the lower bound AJ 7 . By contrast, 
in Fig. [2] the particle starts initially in a nonstationary 
Gaussian state p(x,0) = (2tt ct^)^ 1 / 2 ) cxp(-o; 2 /( 2 f J o))- 
We observe that even for long driving times r, the mean 
work never reaches AJ 7 , due to the nonstatic relaxation of 
the initial state. In the absence of driving, the inset shows 
that the Lyapunov function J- decays monotonously. 

Conclusions. Two state functions can be denned for 
arbitrary nonequilibrium states beyond linear response: 
the information entropy % and the information free en- 
ergy JF. Both can be directly related to measurable ther- 
modynamic quantities like (excess) work and heat for 
quasistatic processes that start in equilibrium (nonequi- 
librium) steady states. For arbitrary transformations, 
they seem to loose their thermodynamic interpretation. 
This is especially true for quasistatic transformations 
that start in nonstationary states. However, they never- 
theless provide useful bounds for the mean (excess) work 
and heat exchanged. In particular, the information free 
energy is a natural generalization of the equilibrium and 
the steady state free energy to which it reduces in the 
appropriate limits. We have shown that it naturally ap- 
pears in the nonequilibrium extensions of the maximum 
work theorem with or without feedback. Moreover, since 
it decays monotonously to steady states at all times, it 
is a nonequilibrium Lyapunov function. Our results may 
also be extended to other types of dynamics [Zl (3Ql [31] . 
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